Spatial Coding and information retrieval in multimodal documents
نویسندگان
چکیده
In this paper we study the effects of Spatial Coding (Murray, Kennedy, 1987; Kennedy, 1992) on the way readers process semantic information while searching for a target in multimodal (textimage) documents. Our aim is to understand how spatial indexes are used by readers in information retrieval tasks, and what are the interactions between semantic distance and spatial distance. We present an eye tracking experiment in which 24 participants were asked to search for information in simple multimodal digital documents while we manipulated their ability to associate spatial indexes to semantic content and changed the location of the target text between the inspection and the search phases. The results show that participants’ performances worsened with the increasing of the semantic distance between the target text and the text that took its place. Results also show that readers can rapidly adapt their information searching habits to the spatial organization of the document.
منابع مشابه
Public Transport Ontology for Passenger Information Retrieval
Passenger information aims at improving the user-friendliness of public transport systems while influencing passenger route choices to satisfy transit user’s travel requirements. The integration of transit information from multiple agencies is a major challenge in implementation of multi-modal passenger information systems. The problem of information sharing is further compounded by the multi-l...
متن کاملXQuery adaptation for multimodal retrieval of multimedia documents
Recent years witness a phenomenal growth of multimedia data in various modalities, such as image, video, audio, and graphic, which poses a challenge of finding an efficient information retrieval technology. Rather than monotonous, single-modal information, users would like to have a multimodal system to query multimedia documents. In this paper, we present our propositions in two parts, the fir...
متن کاملA Survey of Indexing and Retrieval of Multimodal Documents: Text and Images
A document conveys information using multiple modalities, including text, layout/style and images. For example, journal articles usually have figures to illustrate experimental results, and the title in a journal article usually has a different font size than the body text. Indexing and retrieval using only text is the traditional way of IR (Information Retrieval). With the development of the I...
متن کاملNatural language processing versus content-based image analysis for medical document retrieval
One of the most significant recent advances in health information systems has been the shift from paper to electronic documents. While research on automatic text and image processing has taken separate paths, there is a growing need for joint efforts, particularly for electronic health records and biomedical literature databases. This work aims at comparing text-based versus image-based access ...
متن کاملPrototyping a Vibrato-Aware Query-By-Humming (QBH) Music Information Retrieval System for Mobile Communication Devices: Case of Chromatic Harmonica
Background and Aim: The current research aims at prototyping query-by-humming music information retrieval systems for smart phones. Methods: This multi-method research follows simulation technique from mixed models of the operations research methodology, and the documentary research method, simultaneously. Two chromatic harmonica albums comprised the research population. To achieve the purpose ...
متن کامل